{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Iterators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the last lecture we saw that we could approach iterating over a collection using this concept of `next`.\n",
    "\n",
    "But there were some downsides that did not resolve (yet!):\n",
    "* we cannot use a `for` loop\n",
    "* once we exhaust the iteration (repeatedly calling next), we're essentially done with object. The only way to iterate through it again is to create a new instance of the object."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we are going to look at making our `next` be usable in a for loop."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This idea of using `__next__` and the `StopIteration` exception is exactly what Python does.\n",
    "\n",
    "So, somehow we need to tell Python that the object we are dealing with can be used with `next`.\n",
    "\n",
    "To do so, we create an `iterator` type object."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Iterators are objects that implement:\n",
    "* a `__next__` method\n",
    "* an `__iter__` method that simply returns the object itself"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's it - that's all there is to an iterator - two methods, `__iter__` and `__next__`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's go back to our `Squares` example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Squares:\n",
    "    def __init__(self, length):\n",
    "        self.length = length\n",
    "        self.i = 0\n",
    "        \n",
    "    def __iter__(self):\n",
    "        return self\n",
    "    \n",
    "    def __next__(self):\n",
    "        if self.i >= self.length:\n",
    "            raise StopIteration\n",
    "        else:\n",
    "            result = self.i ** 2\n",
    "            self.i += 1\n",
    "            return result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can still call `next`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sq = Squares(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "print(next(sq))\n",
    "print(next(sq))\n",
    "print(next(sq))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course, our iterator still suffers from not being able to \"reset\" it - we just have to create a new instance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sq = Squares(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But now, we can also use a `for` loop:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "4\n",
      "9\n",
      "16\n"
     ]
    }
   ],
   "source": [
    "for item in sq:\n",
    "    print(item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now `sq` is **exhausted**, so if we try to loop through again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for item in sq:\n",
    "    print(item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get nothing..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All we need to do is create a new iterator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sq = Squares(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "4\n",
      "9\n",
      "16\n"
     ]
    }
   ],
   "source": [
    "for item in sq:\n",
    "    print(item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just like Python's built-in `next` function calls our `__next__` method, Python has a built-in function `iter` which calls the `__iter__` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sq = Squares(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1965579635736"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "id(sq)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1965579635736"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "id(sq.__iter__())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1965579635736"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "id(iter(sq))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And of course we can also use a list comprehension on our iterator object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sq = Squares(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 4, 16]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[item for item in sq if item%2==0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can even use any function that requires an iterable as an argument (iterators are iterable):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sq = Squares(5)\n",
    "list(enumerate(sq))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But of course we have to be careful, our iterator was exhausted, so if try that again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(enumerate(sq))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we get an empty list - instead we have to create a new iterator first:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sq = Squares(5)\n",
    "list(enumerate(sq))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can even use the `sorted` method on it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[16, 9, 4, 1, 0]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sq = Squares(5)\n",
    "sorted(sq, reverse=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Python Iterators Summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Iterators are objects that implement the `__iter__` and `__next__` methods.\n",
    "\n",
    "The `__iter__` method of an iterator just returns itself.\n",
    "\n",
    "Once we fully iterate over an iterator, the iterator is **exhausted** and we can no longer use it for iteration purposes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The way Python applies a `for` loop to an iterator object is basically what we saw with the `while` loop and the `StopIteration` exception."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "4\n",
      "9\n",
      "16\n"
     ]
    }
   ],
   "source": [
    "sq = Squares(5)\n",
    "while True:\n",
    "    try:\n",
    "        print(next(sq))\n",
    "    except StopIteration:\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In fact we can easily see this by tweaking our iterator a bit:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Squares:\n",
    "    def __init__(self, length):\n",
    "        self.length = length\n",
    "        self.i = 0\n",
    "        \n",
    "    def __iter__(self):\n",
    "        print('calling __iter__')\n",
    "        return self\n",
    "    \n",
    "    def __next__(self):\n",
    "        print('calling __next__')\n",
    "        if self.i >= self.length:\n",
    "            raise StopIteration\n",
    "        else:\n",
    "            result = self.i ** 2\n",
    "            self.i += 1\n",
    "            return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sq = Squares(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "calling __iter__\n",
      "calling __next__\n",
      "0\n",
      "calling __next__\n",
      "1\n",
      "calling __next__\n",
      "4\n",
      "calling __next__\n",
      "9\n",
      "calling __next__\n",
      "16\n",
      "calling __next__\n"
     ]
    }
   ],
   "source": [
    "for i in sq:\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see Python calls `__next__` (and stops once a `StopIteration` exception is raised).\n",
    "\n",
    "But you'll notice that it also called the `__iter__` method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In fact we'll see this happening in other places too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "calling __iter__\n",
      "calling __next__\n",
      "calling __next__\n",
      "calling __next__\n",
      "calling __next__\n",
      "calling __next__\n",
      "calling __next__\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0, 4, 16]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sq = Squares(5)\n",
    "[item for item in sq if item%2==0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "calling __iter__\n",
      "calling __next__\n",
      "calling __next__\n",
      "calling __next__\n",
      "calling __next__\n",
      "calling __next__\n",
      "calling __next__\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sq = Squares(5)\n",
    "list(enumerate(sq))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "calling __iter__\n",
      "calling __next__\n",
      "calling __next__\n",
      "calling __next__\n",
      "calling __next__\n",
      "calling __next__\n",
      "calling __next__\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[16, 9, 4, 1, 0]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sq = Squares(5)\n",
    "sorted(sq, reverse=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why is `__iter__` being called? After all, it just returns itself!\n",
    "\n",
    "That's the topic of the next lecture!\n",
    "\n",
    "But let's see how we can mimic what Python is doing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "calling __iter__\n",
      "1965579704808 1965579704808\n",
      "calling __next__\n",
      "0\n",
      "calling __next__\n",
      "1\n",
      "calling __next__\n",
      "4\n",
      "calling __next__\n",
      "9\n",
      "calling __next__\n",
      "16\n",
      "calling __next__\n"
     ]
    }
   ],
   "source": [
    "sq = Squares(5)\n",
    "sq_iterator = iter(sq)\n",
    "print(id(sq), id(sq_iterator))\n",
    "while True:\n",
    "    try:\n",
    "        item = next(sq_iterator)\n",
    "        print(item)\n",
    "    except StopIteration:\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, we first request an iterator from `sq` using the `iter` function, and then we iterate using the returned iterator. In the case of an iterator, the `iter` function just gets the iterator itself back."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
